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Abstract 

We consider a pattern of neutrino masses in which there is an approximate mass 
degeneracy between the two mass eigenstates most coupled to the and v T flavour 
eigenstates. Earth-matter effects can lift this degeneracy and induce an effectively 
maximal mixing between these two generations. This occurs if f e 's contain comparable 
admixtures of the degenerate eigenstates, even rather small ones. This provides an 
explanation of the atmospheric neutrino anomaly in which the ab initio introduction 
of a large mixing angle is not required. To test this possibility we perform a novel and 
detailed analysis of the 52 kiloton-year SuperKamiokande data, and we find that in a 
large region of parameter space the corresponding confidence levels are excellent. The 
most recent results from the Chooz reactor experiment, however, severely curtail this 
region, so that the conventional scenario with nearly maximal mixing angles -which 
we also analyse in detail- is supported by the data. 
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1 Introduction 



The results of the SuperKamiokande collaboration (SK) on the atmospheric neutrino 
deficit ||] can be explained in terms of neutrino oscillations It is natural to anal- 
yse the data in the context of the most general mixing pattern of three neutrinos, since 
that is their known number. Three generations are necessary if oscillations are to explain 
the atmospheric and solar ||] anomalies: a scheme with only two neutrinos cannot account 
for both effects. 

Let U, with (u e , v^, v T ) T = U-{u\, i>2, f3) T , be the Cabibbo-Kobayashi-Maskawa (CKM) 
matrix in its most conventional parametrization, reviewed by the Particle Data Group [II: 





U = U 23 U 13 U 12 = C 23 S23 1 -S12 C12 (1) 



with s\2 = sin 0i2, and similarly for the other sines and cosines. Several groups have 
performed analyses of atmospheric and solar data in terms of three-neutrino mixing ||, 
as described by Eq. ([!]), or including sterile neutrinos ||. It is common to these studies 
to restrict to a "minimal scheme", in which the mass square difference relevant to atmo- 
spheric oscillations dominates over the one relevant to solar neutrinos: Ara^ ^> Am 2 2 . In 
this scenario, the number of parameters describing oscillations at terrestrial distances is 
reduced to three: S13, S23 and Ara^, while those most relevant to solar neutrinos are: S12, 
S13 and Am 2 2 . The best fit of the atmospheric data [0] is: 

Am|j ~ 2.8 x 1(T 3 eV 2 , sin 2 (2# 2 3) ~ 1, s 2 3 ~ 2 x 1(T 2 . (2) 

The angle #23 — 7r/4 is close to maximal, to explain the dearth of muons in SK. 

The situation for solar neutrino oscillations is less definite The combined solar 
experiments allow for three different regions of parameter space. The solar deficit can be 
interpreted either as MSW (matter enhanced) oscillations Q with an angle 612 that can be 
large or small, or as nearly-maximal vacuum oscillations, #12 — vr/4. The corresponding 
mass differences -Am 2 2 = 10 -6 to 10 -4 eV 2 , or some 10 -10 eV 2 - are significantly below 
the range deduced from atmospheric observations, giving support to the minimal scheme. 
If Ato 2 2 ~ O(10 -3 ) eV 2 , it can have non-negligible effects on atmospheric data that have 
been recently studied [1C]. 



As is well-known, the field of neutrino oscillations is permeated by a tally of implausible 
facts and coincidences. Oscillations over a distance L occur if Am 2 L/E u ~ 1. In the 
various data samples used by the SK collaboration, the average neutrino energies are 
roughly 1, 10 and 100 GeV, so that for the value of Atn| in Eq. (||), the "right" distances 
to measure an effect are 280, 2800 and 28000 km: the size of our planet and the energies 
in the cosmic ray spectrum have been chosen snugly. Something entirely similar can be 
said about the low-mass or "just-so" solution to the solar neutrino problem. Moreover, the 
solar and atmospheric neutrino "anomalies" could have been observed only if the effects are 
large. This requires surprisingly large mixing angles, except for the "small- angle" MSW 
solution to the solar neutrino problem, for which the ambient matter elegantly enhances 
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the effect of a small vacuum mixing. Considerable theoretical effort has been invested in 
arguing that large neutrino mixings are natural, as small quark mixings are believed to 
be. 

A final peculiarity of the observed atmospheric neutrino oscillations involves the matter 
contribution to the effective squared mass of electron neutrinos: 

A = 2 V2 G F E v n e , (3) 

where n e is the electron number density in the Earth. For a typical average terrestrial 
density of 5 g/cm 3 and E v = 10 GeV, A ~ 3.7 x 10 -3 eV 2 , again in the ballpark of Eq. (0) . 
This last coincidence suggests the existence of a "small-angle solution" to the atmospheric 
neutrino problem. 

In the "small-angle" scheme we study, the large observed disappearance is gen- 
erated in the following way. Let V2 and v% be almost degenerate, and heavier than v\. 
When neutrinos traverse the Earth, the degeneracy is lifted by matter effects, enhancing 
Vpr^T oscillations. The consequent transitions are maximal if the electron neutrino has 
comparable vacuum admixtures of the degenerate mass eigenstates, U(e2) ~ £7(e3), even 
if these quantities are not large. 

We find that an explanation of the atmospheric neutrino anomaly not involving nearly- 
maximal neutrino mixing indeed exists, but is disfavoured by the complementary informa- 
tion from reactor neutrinos. For the current experimental situation, large neutrino mixing 
angles seem to be unavoidable. 

The titles of the chapters and appendices specify the structure of this paper. 



2 Apparent large mixings induced by matter effects 
2.1 From small to large angles 

In a three-family scenario, let one neutrino mass eigenstate be much lighter than the other 
nearly-degenerate two. Their squared mass matrix can be written as: 

MJT ~ Diag [fi 2 ; m 2 + m 2 + fi 2 ] , (4) 

where K stands for the "kinetic" eigenbasis (as opposed to the flavour basis), vac is for 
vacuum, and [i 2 <^m 2 . The degree of degeneracy assumed for the two heavier neutrinos 
embodies three conditions: their mass difference Am^ = fi 2 — fi 2 , is much smaller than 
their common mass scale m 2 ; it is also small enough not to induce observable oscillations 
over terrestrial distances (Am 2 3 L/E v <^l for the relevant energies and lengths of travel); 
and it is smaller than the effective mass excess induced on electron neutrinos by matter 
effects, Am^ <C A, with A as in Eq. (|3|). De facto, these conditions simply amount to 
Am 2 ,^ <^ 10~ 4 eV 2 . In practice we can set fif = in the analysis of oscillations over 
terrestrial baselines. 

For the mass pattern of Eq. (||), one of the three neutrino mixing angles and the 
CP-violating phase of the CKM matrix U are unobservable. This is readily checked. 
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Parametrize the CKM matrix in the unconventional order U = ^12^13^23 ; as follows: 






v = r l2 r, :; ( 2:s = \ <■,, o | | o i u j [ o C23 ^ S23e i<5 | ( 5) 

-S23 e~ tS C 23 

with S12 = sin(#i2), etc. The vacuum mass matrix in flavour space M™ c = UM^ C U^ does 
not depend on S23, or on 5. The mixing matrix is effectively reduced to U = U\ 2 Ui 3 . 

In the approximation we are discussing, m 2 is the only relevant mass-scale difference 
and the vacuum transition probabilities between different neutrino flavours are: 



P{v e - 
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c 12 
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(6) 

The probabilities P{y e — ► f^) and P(^ e — ► i/ r ) are quadratically suppressed for small S12 
and S13, while P{yn — > is quartically suppressed. The situation in matter, however, is 
drastically different. 

It is convenient to work in the "kinetic" basis wherein Mj^ c is diagonal. The effect of 
matter is fully encrypted in a modification of the squared mass matrix: 

(A+B 0\ 
M% at = M^ ac + C/t B )U, (7) 




where, as is well known, B arises from flavour-universal forward-scattering neutral current 
interactions while A, given by Eq. (H), arises from the charged-current contribution specific 
to v e e and v e e scattering. 

To illustrate how matter effects lift the vacuum degeneracy between two mass eigen- 
states, we diagonalize M^ at to first order in A/m 2 , temporarily assumed to be small (the 
exact formulae, used in the numerical results, are presented in Appendix A). To order zero 
in this expansion there are two equal eigenvalues, so that we must follow the usual rules 
of degenerate perturbation theory. Write M^ at = M' ' + MW with: 

/0 

AfM = m 2 + As 2 12 

\0 ^Ci 2 Si 2 Si3 

/ Acj 2 c 2 3 Ac 12 c 13 s 12 Aci 3 c^ 2 si3\ 
M' 1 ' = Ac 12 c 13 s 12 (8) 

\Ac 13 c 2 l2 s l3 / 

where we have subtracted the common entry B, which plays no role in neutrino mixing. 
We must diagonalize M' ' exactly to lift the degeneracy in M vac , then the second term 
can be consistently treated in perturbation theory. 
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To order A/m , the flavour and kinetic mass matrices, and Mj^ , are: 



where 



M mat 

M% at 



U 12 U 13 U mat Mr t ul at ul 3 Ul 2 ; 



Diag A c 2 12 cl 3 ; m 2 ; m 2 + A (s\ 2 + cf 2 sj 3 ) 
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and the sine of the mixing angle in matter is: 



mat 
s 23 



S 12 / J S 2 l2 + cj 2 S 2 



13 



(9) 
(10) 

(11) 
(12) 



There are two important points to notice. First, instead of one mass difference as in 
vacuum, we have two: 



Am? 2 
Ara a 



m 

A (s 2 2 + c 12 s 13) > 



(13) 



one of 0(m 2 ), the other of 0(A), the matter-induced mass. Secondly, the matrix U ma t 
plays the same role as the mixing matrix U23 in the generic mixing scenario of three 
neutrinos. The effect of matter is simply to split the degenerate eigenvalues and induce 



the effective angle of Eq. (12). The crucial point is that this angle can be large 



even if s±2 and S13 are small. To this order in A/m 2 , the condition for maximal mixing 
is sis = S12/C12; in a parametrization-independent language, this is equivalent to the 
requirement that the mixing between the second and third eigenstates with the electron 
flavour state be the same: U(e2) = U(e3). 



2.2 Oscillation probabilities 

For the Earth's electron density appearing in A, Eq. (0), it is a good approximation [11] 
to consider a piecewise-constant density profile: a negligible density for neutrinos travers- 
ing the atmosphere, p m = 5 g/cm 3 for the mantle, and p c = 11 g/cm 3 for a core of 
3500 km radius. The matter-induced squared "mass" can then be expressed as A = 
2\[2GFE v (n e (c v )) , where the electron density is averaged over the neutrino trajectory, 
and c v is the cosine of its zenith angle. The oscillation probabilities depend on the two 
mass differences of Eq. (|1|) and have the same form as the CP-conserving part of the 
general three-flavour vacuum-transition probabilities: 



P^fr,*) = -4 £ MWl% sin 2 f' Am hf ((g I ■ (ID 



k>3 \ V 



with W 3 a p = [UajUpjU^Upk], and U = UnUi3U ma t- The distance L{c u ) is: 

L{cu) = R*(yfO- + l/R®) 2 -si- c„), (15) 



5 



where is the radius of the Earth and I ~ 15 km is the typical height at which primary 
cosmic rays interact in the atmosphere. 

Consider the <-> v T entry of Eqs. (|l4|): 



D ftp \ ■ 2/r, Q mat\ ■ 2 ( ^ m 23 L(c u ) \ 2 2 \ 

n^ T (^,c w ) = sm (2 6» 23 ) sin [ — ) + 0(s 12 ,s 13 ). (16) 



V 



Even if s\i and S13 are small, P v Vr can be maximal, since #23°* — vr/4 for s%2 — S13. 
In the limit sx2 5 S13 — ► 0, Am| — > and the oscillation probability vanishes: there cannot 
be oscillations if all CKM angles are zero. 

Large Vn — > u T oscillations take place for s\2, S13 small, but bounded from below by the 
condition Ara| R®/E u ~ 0(1). For this parameter range one should still check the size 
of v e — ► Vp, v T transitions, which are not observed. This turns out not to be a problem for 
the atmospheric anomaly, because the v^/u e flux ratio is close to 2, and in the region of 
maximal mixing and small vacuum angles P{y e — > Un) ~ P(v e — > v T ). Consequently, the 
number of disappearing v e can be compensated by the number of v^s that oscillate into 
v e s J 12]. But a large v e disappearance probability can lead to a violation of the stringent 
bounds imposed by the Chooz experiment jH| [14]. We shall see that Chooz, but not 
SK, disfavours our small angle scenario. 



2.3 Relation to the conventional scenario 

The degenerate-neutrino scenario involves only one vacuum mass difference in the de- 
scription of terrestrial experiments. This is also the case in the scenarios considered in 
most previous analyses of atmospheric data, even with three families || , but with a single 
dominant mass difference. 

As it turns out, the degenerate scenario is exactly equivalent to the conventional one 
of H| with Am| 3 = — m? (in vacuum oscillations the sign of this difference would be 
unobservable) . To see this equivalence explicitly, it suffices to consider their vacuum 
CKM matrix, which is written in the customary order U = ^23(523) ^13(^13)) an d to 
obtain from it the matrix of Eq. (|5|) via the substitutions: 

s 2 2 

-2 _ A 12 L 13 
S 23 ~ s 2 T ,2 „2 ' 
b 12 ^ 6 13 L 12 

4 = 4 2 c 2 13 . (17) 

Note that small mixing angles in the degenerate parametrization may correspond to 
large mixings in the conventional one. In particular, a region of arbitrarily small mix- 
ing angles sf 2 ,Si3 of our mass-degenerate scenario is mapped to a domain around the 
values S23 ~ 1/2 and sf 3 ~ 1 of the conventional parametrization. The most "natural" 
parametrizations are the ones in which the rotation matrices Uij act on the mass eigen- 
states in order of decreasing degeneracy. The conventional parametrization, used in the 
Particle Data Book is natural for the quark sector with its hierarchical mass splitting, 
but not necessarily for the lepton sector. The parametrization we use, Eq. (|5|), is natural 
for the partially-degenerate mass pattern that we are considering flDJ. 
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As we saw in the previous subsection, the presence of degenerate eigenstates in vacuum 
can lead to large transition probabilities in matter. The enhacement of transition proba- 
bilities in matter in the context of three-family mixing with two degenerate neutrinos has 
been discussed before in [|l(| and, in the context of the three-maximal mixing model, in 
[12]. The parametrization we use here clarifies the origin and generality of the effect. 



3 Zenith angle and energy distribution of the SK events 

The data of the SK collaboration, as well as their Monte Carlo expectations for the case in 
which there are no neutrino oscillations, are binned in the azimuthal angle of the observed 
electrons and muons, and in their energy (in the case of muons the level of containment 
within the detector also distinguishes different data samples). To reproduce these results 
one must convolute neutrino fluxes and survival probabilities with charged-current differ- 
ential cross sections and implement various efficiencies and cuts. This being an elaborate 
procedure, in Appendix D we check our results by reproducing the no-oscillation Monte 
Carlo results of SK, as well as the neutrino "parent energy" spectrum: the azimuthally 
averaged neutrino flux weighted with the integrated neutrino cross section and with the 
efficiencies of the various data samples. 

In the rest of this section we review how the data are binned, we specify our procedure, 
and we analyze the fits to the conventional oscillation scenario, as well as to our mass- 
degenerate alternative. 

3.1 The data samples 

The SK collaboration chooses to bin the observed charged-lepton energies in a few samples. 
The electron candidates are subdivided into sub-GeV (sgev) and multi-GeV (mgev). The 
muon candidates are distinguished as sgev and mgev, partially and fully contained (PC, 
FC), and through-going (thru). To set apart these categories, we introduce selection 
functions Th Si j(^,q), with s = sgev, mgev and I = e, /i, that depend on the energy, £/, 
and on the cosine, q, of the azimuthal angle of the outgoing lepton (q = 1 is vertically 
down-going). In computing the number of events, these selection functions will weight the 
product of neutrino flux and cross-section. For the sgev events, 

Th sgcVii (^) = @[E th>l - Ei] S[Ei - £ min ,i], (18) 

with E thAfl) = 1.33(1.4) GeV, EUmefc) = 100(200) MeV for e(p) respectively. For the 
mgev electron events, Th mgeVje (^) = @[Ei — i?th,e]- 

For the mgev muons, we must distinguish between partially and fully contained events: 
Th mgev _pc, At (£' At , c M ) = 6[£yj — -Eth,/z] PC(E /1 ,c fl ) , 

Th mgev _ FClM (£ M , qj = Q[E„ - E th J FC(£ M , qj , (19) 

where the functions FC(E fl , c^), PC(E fl , c^) measure the fraction of the total fiducial 
volume in which a neutrino interaction could produce a fi with energy E^ and zenith 
direction c M that either stops before exiting the detector (FC) or escapes (PC). We have 
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explicitly constructed FC(E^, c M ) and PC(E fl , c M ) using the shape and size of the detector 
and the \i range in water, R W (E^), as a function of energy, and describe this in Appendix 
C. 

We have also devised a through-going muon selection function Ththru,^ fy). The 
effective target mass of the rock surrounding SK depends on energy via the muon range 
in water and in rock, R r (E^). The observed muon energy is required to be greater than 
E ' in = 1.6 GeV, implying that its trajectory must be longer than 7 m. The function 
Th t hru,/i must account for the detector's effective area for such tracks, A(E fl ,c^ l ), which 
depends, via the muon range, on the muon energy as it enters the detector, and on its 
zenith angle. Furthermore, the selection function for through-going muons must take into 
account that their flux, as given by SK, is defined as the number of events divided by the 
effective area for a muon of energy E' min [|17]] . All in all: 

Th thru ,^, c M ) ee [Rr(E,) - R r (E' mm )] ^r C "\ - (20) 
This effective area is given in Appendix C. 



3.2 Number of events 

Let d$> u (E u ,c u )/dE u dc u and d<& u (E u , c u ) /dE u dc u be the atmospheric neutrino fluxes of 
v = v e ,v„ and their antiparticles, with E u the neutrino energy and c u its zenith angle 
(we use the Bartol code [18] of atmospheric neutrino fluxes at the Kamiokande site). 
Let da(E u , Ei, cp)/dEi dcp and da(E u , Ei,cp)/dE[ dcp be the neutrino and antineutrino 
charged-current cross sections, which depend on E v , on the outgoing lepton energy Ei, 
and on the cosine of the scattering angle between the two particles, cp. In Appendix B, we 
discuss in detail the cross sections used in our analysis. The zenith angle of the outgoing 
lepton q, which is the measured quantity, is a function of c u , cp, and of the azimuthal 
angle, cp, of the outcoming lepton in the target rest frame. 

Let Ngi (c) , N° s f (c) be the expected number of charged-current events in the sample s 
(s = sgev, mgev-pc, mgev-fc, thru) for I = e, /i and in the bin in zenith cosine with central 
value c, for the no-oscillation (0) and oscillation (osc) hypotheses. For the sgev and mgev 
samples, the theoretical prediction is given by: 

N^(c) = K Sjl J dE u dc v dEi dcp d<\> Th s (^, a) (9[q -c + S}- 9[q - c - 5}) 

E 



cr(E u ,Ei,cp) d ^ u ' P u , v (E u ,c u ) + v(E v ,E h cp) P v > v (E v ,c u 



(21) 



for the oscillation hypothesis, with P u i u (E l ,,c u ) and P u ' u (E u ,c u ) the oscillation probabil- 
ities from flavour v' to flavour v for neutrinos and antineutrinos respectively. To obtain 
iV , take P v i v = P v i v = b u iy. The Q functions in Eq. ( pi] ) express the constraint that q 
be in the bin with central value c and width 25 = 0.4, the binning used in SK for the 
sgev and mgev samples. Finally, K s i are normalization constants, which ensure that the 
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total number of events in each sample is the same as in the SK Monte Carlo data for the 
non-oscillation hypothesis. By choosing these factors by hand, we skirt the question of 
efficiencies for electron or muon detection and for single- or multiple-ring events: all we 
need to assume is that they are roughly constant within a given data sample, which we 
believe to be the case. We neglect the cross-talk between different samples. 

For the flux of through-going muons, we have: 
^thm(c) = K thru J d<j)dc p dE^dc u dE u Th thrU)/ ,(£ M , C(U ) (9^ -c+S] — Q[c^ -c-S]) 

<t(K, En, cp) Pv'^iK, c v ) + a(E„, cp) P u , Uti (E u ,c u ) 

(22) 

for the case with oscillations. The width of the zenith angle bins in this sample is 28 = 0.1. 



3.3 Results of the analysis of the SK data 

We have performed a x 2 analysis of the oscillation hypothesis in our mass-degenerate 
scenario, for both signs of m 2 , using the full 52 kiloton-year data sample gathered by the 
Super-Kamiokande collaboration in 848 days of exposure JO]] . The case of negative m 2 is 
exactly equivalent to the conventional scenario considered in j7|, as shown in Section 2. 
The measured quantities are the 30 zenith angle bins measured by SK in the five types 
of data samples. The choice of an error correlation matrix is non-trivial, as there are 
large theoretical uncertainties in the input neutrino flux, which induce large correlations 
between the errors in the different measured quantities. We have constructed the error 
correlation matrix in the same way as the authors of Q, to whom we refer for details. To 
gauge the incidence of these "systematic" errors on the results, we have also performed 
the analysis with only statistical uncertainties. 

In Fig. |l] we show, in the plane Si2 — S13 and for several positive values of m 2 , the 
contour lines delimiting the allowed regions at 68.5 and 99% confidence. In Fig. |||the same 
information is displayed for negative m 2 . The region of maximal mixing in the conventional 
parametrization corresponds to values S12 ~ 1 and S13 ~ l/v^ of our parametrization. 
This region is favoured for the smaller allowed values of m 2 : the top two rows of Fig. |l]. At 
the larger values of m 2 , however, the contours extend largely to a region with significantly 
smaller vacuum angles, the oscillation probabilities being enhanced by matter effects. 
We draw for comparison the line corresponding to maximal mixing in the perturbative 
approximation of Eq. (12), valid for the larger m 2 values. The allowed regions at small 
angles are close to this line, as expected. At values of m 2 smaller than those shown in the 
figure, the allowed regions shrink around the conventional maximal-mixing solution. 

The minimum x 2 is obtained for \m 2 \ ~ 3.5 x 10 -3 eV 2 , independently of whether 
the errors are taken to be purely statistical, or estimates of flux uncertainties are also 
included. This result is in good agreement with that found by the SK collaboration 
in a two-family mixing context. We do not find such an agreement on the optimized 
mixing angles. The best-fit angles in our parametrization are s 2 2 ~ 0.42 (0.45) and 
s 2 3 ~ 0.31 (0.33) for positive (negative) m 2 , which in the conventional parametrization 
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correspond to S23 = 0.48 (0.48) and sf 3 = 0.4 (0.37), not so far from the so-called tri- 
maximal mixing model (s 2 2 = 1/2, sf 3 = 1/3) fll2| . The Chooz data, however, disfavour 
these relatively large-m 2 and small-angle solutions. 



4 Chooz Constraints 

The reactor experiment Chooz provides tight upper limits on the v e disappearance prob- 
ability in a domain with Am 2 > 10 -3 eV 2 [13] [14]. This entails very strong strictures on 



P^ e p M and Pu e p r at atmospheric distances. 

We have constrained our analysis of atmospheric data to comply with the Chooz results 
on the ratio R of observed e + events to the number expected in the absence of oscillations: 

p> = JdE u <S>(E v )a(E u )P PePe (E„) 

J dE u <t>(E u ) a{E v ) ' {0) 

where <&(E U ) is the spectrum of neutrinos, obtained by combining, in the appropriate 
proportions 0], the decay spectra of the different isotopes in the Chooz reactors [20]. 



In writing Eq. (|23| ) we have approximated the efficiency as a constant, for lack of better 
information. The cross section, a(E u ), including the threshold effects, has been obtained 
from [ 2 1 1 . For the transition probabilities we can use Eq. (^), since matter effects are 



completely negligible. 

The results of this combined SK-Chooz analysis are shown in Fig. || for positive m 2 . 
The results for negative m 2 shown in Fig. || are very similar. Clearly the Chooz data 
favour the conventional maximal-mixing solution as the only acceptable one. 

In Fig. H], we show the minimum x 2 as a function of mass, for positive m 2 . On the left 
we include theoretical flux errors as in 0], while on the right only statistical uncertainties 
are taken into account. Reassuringly, the theoretical errors have a small incidence on 
the results. In Fig. ^, we show the results for negative m 2 . For both signs of the mass 
difference, the minimum of the x 2 occurs at |m 2 | = 2-2.5 x 10 3 eV 2 , which is slightly 
smaller than the value obtained in the combined analysis of @ (|m 2 | ~ 2.8 x 10 -3 eV 2 ). 
Concerning the mixing amplitudes, we find as the best fit for the important |^2j ]l5| angle 
#13 -in the conventional parametrization- at #13 = 6°, to be compared to 8° found in M. 
However, the \ 2 curve is flat enough for #13 = to be perfectly compatible with the data. 

In Figs. [7| and |^ we show the impressive agreement between the SK zenith angle distri- 
butions and our best-fit oscillation hypothesis, obtained including the Chooz constraint. 

Incidentally, for the trimaximal mixing model we get x 2 = 42 (44) (for 30 degrees of 
freedom: 31 data minus one free parameter m 2 ) for positive (negative) m 2 at |m 2 | = 10 -3 
eV 2 , a mass value for which the Chooz constraint is inoperative. The x 2 rises rapidly for 
larger |m 2 |. The probability that this model is correct is below 10%. 
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5 Conclusions 



The neutrino squared mass difference used to explain the atmospheric neutrino anomaly 
in terms of oscillations is of the same order of magnitude as that induced by Earth matter 
effects, for a typical atmospheric neutrino energy. Triggered by this coincidence, we set 
out to study -in a scheme with three neutrinos and in minute detail- whether or not 
the large depletion of muon neutrinos observed by SK could be due, not to ab initio 
large mixing angles, but to matter-enhanced smaller-angle mixings. Our suspicion turned 
out to be correct: the SK data can be very satisfactorily explained with mixing angles 
that are far from maximal. But the constraints from Chooz on the survival of electron 
antineutrinos disfavour our non-maximal solution. The conclusion that vacuum v„ «-> u T 
neutrino transition probabilities are nearly maximal may be surprising, but it is here to 
stay. 
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Appendix A: Oscillation parameters 

The exact diagonalization of the mixing matrix in Eq. (]?]) results in the effective eigen- 
mass differences: 



Am\ 2 



Am 2 
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m 2 - A + y (m 2 + A) 2 - 4 A m 2 cf 3 cf 2 J , 
-m 2 + A + J (to 2 + A) 2 - 4A m 2 cf 3 c 2 2 



and in a mixing matrix: 

/-(C12C13 - a + )/b + 

Umat — 



Sl3C 12 /Js 2 u + S 2 3 C 2 



12 



V 



12 



(C12C13 - a-)/b- \ 
siz/b- 
s 13 c 12 /b- / 



where 



b± 



-su/b + 
-s 13 ci 2 /b^ 



A + m 2 ± J(m 2 + A) 2 - 4 A m 2 cf 3 cf 2 /(2 A c 12 c 13 ), 



1 + a± - 2 a± ci3 ci2 



(24) 



(25) 



(26) 
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Appendix B: Neutrino cross sections 

The proper treatment of neutrino cross sections in water and rock, at energies rang- 
ing from 100 MeV to hundreds of GeV, is an arduous art. We do not attempt an 
elegant and complete analysis. Instead, we use a treatment that -notwithsanding its 
oversimplifications- is capable of reproducing to an adequate level the observed total 
cross sections and scattering angle distributions, which are the ingredients needed for the 
data analysis. 

We simplify the neutrino cross sections in water by considering only oxygen as a target, 
an isoscalar nucleus for which we ignore shadowing, but not Fermi-motion effects, which 
we treat as in [23|. We also neglect the muon mass. As the SK experimenters do, we build 



a cross section out of three dominant contributions: quasi-elastic {yiM — * IN'), resonant 
one-pion production, and "deep" inelastic. In so doing we ignore the small contribution 
of the "diffractive" domain of relatively high energy, low Q 2 . 

For the quasi-elastic cross section we use the standard expression reviewed in [23], with 



Ma = 1-0 GeV for the mass describing the axial form factor. For one-pion production 
we use Eq. (22) of |24J] for the excitation of the A r *(1236) resonance of spin and isospin 
3/2. We assume that these contributions saturate the cross section for an invariant mass 
of the final hadrons W < 1.4 GeV. Above that value we use a deep-inelastic cross section 
with an exact Callan-Treiman constraint F2 = 2xF\. For the structure functions F2 and 
F3 we use the compilation of (2!| . As an excuse to extend this deep inelastic cross section 
to values of Q 2 as low as 0.4 GeV 2 , we use ^-scaling. This is known to deal correctly with 
the higher-twist target-mass corrections [^6| and to interpolate the resonant contributions 
in the sense of local duality [27]. Alas, the structure functions are not extracted from 



the data using ^-scaling. But the authors of []2a] have shown that, at least in the case of 
electroproduction, a blind a posteriori use of ^-scaling improves the fit to the lower Q 2 data 
(their prescription consists in using the Bjorken-scaling cross-section expressions [Q] and 
the structure functions extracted at higher Q 2 , with the simple and not fully consistent a 
posteriori substitution x — > £ in the argument of the structure functions). 

In Fig. H|, we plot a TOT (E u ) as a function of E u for neutrinos and antineutrinos. The 
curves are a bit below the available data at low energies, no doubt reflecting the absence of 
a calculated diffractive contribution. The SK collaborators, as well as many other authors, 
extend the structure functions down to Q 2 = 0, thereby obtaining a slightly better "fit" 
to the <7 TOT data. Rather than indulging in this inconsistent use of the deep inelastic 
structure functions, we have checked that our results on neutrino-mixing parameters are 
insensitive to this kind of variations of the input. 

An important quantity in the zenith angle analysis is the average scattering angle 



between the lepton and the parent neutrino. In Fig. 1C, we show the average angle for 
quasi-elastic events as a function of the lepton energy. This curve is in perfect agreement 
with that obtained by the SK collaboration |2^, |30| . 

Appendix C: Geometrical acceptances 

The SuperKamiokande detector is a cylinder of height H = 36.2 m and radius R = 16.9 
m, of fiducial volume V = ir (R — 2 m)(H — 4m). Let a point within the detector be 
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labelled by y, the height from the bottom plane; z, the distance from the axis; and x, a 
third cartesian coordinate. Let d(x,y,z) be the minimum distance from a point in the 
detector to its wall, let Sn,Cn,tn be the sine, cosine and tangent of the muon's azimuthal 



angle and let En 
We find: 



0.7 GeV be the energy of a muon giving a 2.6 m track in water. 



2 rR r 2VR 2 -z' 2 r H 
FC(E fl ,c fi ) = - dz dx dy e[d(x,y,z) -2m] 

v Jo Jo Jo 







2VR 2 -z 2 -x-y\t fl \ efy-iU^/JW] 



+ e 



x 



e 



y\U+x-2VB? 



(27) 



2 rR r2VR 2 -z 2 rH 

PC(i? At , c^) = — Q[R w (E fl ) - R w (E min )] dz dx dy 

v Jo Jo Jo 



Q[d{x,y,z) -2m] l@ [2V R 2 - z 2 - x - y |* M |] 6 [R W {E^) |c M | - y] 

+ e Lr™(# m ) |s M | - 2^i? 2 - z 2 + si e [y |^| + x - 2VR 2 - z 



(28) 



where the range of muons in water that we use, R W (E^), can be obtained from the ex- 
pressions in ||]. 

A muon produced with energy En, after travelling a distance / in rock material, has 
an energy E'^ = R~ 1 [R r (E fl ) — I]; its remaining range in water is ^(E^, I) = R W (E'^). The 
effective area for through-going muons of Eq. (^) is then given by 



A(-E M ,c M ) 



*(^)-^KJ j r R d 
Rr(Efj,) - R r (E' min ) Jo Jo 



H 



dy 9 



l^E^l) -Mm 

e 



2 VR 2 - z 2 [H-y] 



dx e 



^(^,0 -Min 



'2VR 2 ~z 2 -x H 



e 



Min 



7m 



(29) 



where for the range of muons in rock R r {E Uj ), we have used the results in [17]. 

All of the above expressions can be integrated explicitly, but the analytical results are 
not brief, or useful. 
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Appendix D: Comparison with the SK Monte Carlo 

We have not included in this analysis any non-geometrical detection efficiencies, as 
discussed in Section 3. We have normalized the number of events in each data sample 
to the corresponding number in the SK Monte Carlo for the non-oscillation hypothesis. 
This is tantamount to the use of an efficiency function which is not a function of energy 
within each sample. A non-trivial check that this is indeed a sensible approximation is to 
compare the parent neutrino energy distributions in the different data samples with those 
worked out by the SK team. These distributions are defined as the azimuthally averaged 
neutrino flux weighted with the integrated neutrino cross section and with the selection 
function for the various sgev and mgev data samples: 



P s ,l{E u ) oc J dc u dE t dcp d(j) Th. s> x(E u q) 



s d& u d$ u 

a{E ^ cp) dEZdV v + El ^ ] dKd^ 



All of the symbols in this expression have already been defined. Our results, shown in 
Figs. [Tl] and|l^, are in good agreement with those obtained by SK [pi] ]. 



In Figs. |13| and |14j we compare the zenith angle dependence obtained in our calculation 
for the non-oscillation hypothesis with the predictions of the SK Monte Carlo [)30(] . The 
agreement is again rather good. 
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Figure 3: 68.5, 99% CL intervals allowed by SK and Chooz data for different values of m 2 . 
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Figure 5: Minimum \ 2 as a function of m 2 . The solid line corresponds to SK+Chooz 
data, while the dashed line includes only SK data. The curves on the left plot include 
theoretical uncertainties as in Q, while the one on the right includes only statistical errors. 
The horizontal lines correspond to 99% CL intervals for three degrees of freedom. 
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Figure 7: Zenith angle distributions of e and SuperKamiokande samples (squares) com- 
pared with the best-fit oscillation hypothesis: m 2 = 2 x 10~ 3 eV 2 and close to maximal 
mixing. The errors shown are only statistical. 
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Figure 8: Zenith angle distribution of through-going muons (squares) compared with the 
best fit oscillation hypothesis: m 2 = 2x 10~ 3 eV 2 and close to maximal mixing. 
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Figure 9: Total cross section over neutrino energy for v and v charged-current scattering 
on an isoscalar target. 
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Figure 10: Average scattering angle between the parent neutrino and the lepton for quasi- 
elastic events. 
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Figure 11: Parent v energies for ju-like (sgev, FC-mgev, PC-mgev) and e-like (sgev and 
mgev) events. 




Figure 12: Parent v energies for through-going muons. 
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Figure 13: Zenith-angle distributions of e, /j, sgev and mgev samples. The squares are the 
SK Monte Carlo results and the circles are our predictions, both for the no-oscillation 
hypothesis. The total areas under the curves are normalized to be the same. 
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Figure 14: Zenith-angle distributions of through-going muons for the non-oscillation hy- 
pothesis. The squares are SK Monte Carlo results and the circles are our predictions. 
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